Aggregation and Emergence in Agent-Based 
Models: A Markov Chain Approach 
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Abstract 

We analyze the dynamics of agent -based models (ABMs) from a Markovian 
perspective and derive explicit statements about the possibility of linking a 
microscopic agent model to the dynamical processes of macroscopic observables 
that are useful for a precise understanding of the model dynamics. In this 
way the dynamics of collective variables may be studied, and a description of 
macro dynamics as emergent properties of micro dynamics, in particular during 
transient times, is possible. 



1 Introduction 

Our work is a contribution to interweaving two lines of research that have devel- 
oped in almost separate ways: Markov chains and agent-based models (ABMs). 
The former represents the simplest form of a stochastic process while the latter puts 
a strong emphasis on heterogeneity and social interactions. The usefulness of the 
Markov c hain formalism i n the analysis of more sophisticated ABMs has been dis- 
cussed by llzquierdo et al.l (120091 ) . who look at 10 well-known social simulation models 
by representing them as a time-hom ogeneous Markov chain. Among these models 
are the Schellin g segregation m odel (jSchellingl ( 1197lh ). the Ax elrod model of cul- 
tural dynamics (lAxelrodl (119971)') and the sugarsc ape model from Epstein and Axtell 
(119961 ) . The main idea of llzquierdo et al.l ( 120091 ) is to consider all possible configu- 
rations of the system as the state space of the Markov chain. Despite the fact that 
all the information of the dynamics on the ABM is encoded in a Markov chain, it is 
difficult to learn directly from this fact, due to the huge dimension of the configura- 
tion space and its corresponding Markov transition matrix. The work of Izquierdo 
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and co-workers mainly relies on numerical computations to estimate the stochastic 
transition matrices of the models. 

Consider an ABM defined by a set N of agents, each one characterized by indi- 
vidual attributes that are taken from a finite list of possibilities. We denote the set of 
possible attributes by S and we call the configuration space £ the set of all possible 
combinations of attributes of the agents, i.e., S = S N . This also incorporates models 
where agents move on a lattice (e.g., in the sugarscape model) because we can treat 
the sites as "agents" and use an attribute to encode whether a site is occupied or 
not. The updating process of the attributes of the agents at each time step typically 
consists of two parts. First, a random choice of a subset of agents is made according 
to some probability distribution uj. Then the attributes of the agents are updated 
according to a rule, which depends on the subset of agents selected at this time. With 
this specification, ABMs can be represented by a so-called random map representa- 



tion w hich may be taken as an equivalent definition of a Markov chain (jLevin et al. 



( 2009h ). Hence, ABMs are Markov chains on S with a transition matrix P. For a 
class of ABMs we can compute transition probabilities P(x, y) for any pair i,)/GS 
of agent configurations. We refer to the process (X, P) as micro chain. 

When performing simulations of an ABM we are actually not interested in all the 
dynamical details but rather in the behavior of variables at the macroscopic level 
(such as average opinion, number of communities, etc.). The formulation of an ABM 
as a Markov chain (X, P) enables the development of a mathematical framework for 
linking the micro-description of an ABM to a macro-description of interest. Namely, 
from the Markov chain perspective, the transition from the micro to the macro level 
is a projection of the Markov chain with state space S onto a new state space X by 
means of a (projection) map II from X to X. The meaning of the projection II is to 
lump sets of micro configurations in X according to the macro property of interest 
in such a way that, for each X G X, all the configurations of X in II -1 (X) share the 
same property. 

The price to pay in pa s sing from the micro to themacro dynamics in this sense 
(IKemeny and Snelll (119761 ). IChazottes and Ugaldd ( 120031 )) is that the projected sys- 
tem is, in general, no longer a Markov chain: long memory (even infinite) may 
appear in the projected sy stem. In particular, well known conditions for lumpability 
(IKemeny and Snelll (119761 )) make it possible to decide whether the macro model is 
still Markov. Conversely, this setting can also provide a suitable framework to un- 
derstand how aggregation may lead to the emergence of long range memory effects. 



2 The Model 



We ill u strate these ideas a t the example of the Voter Model (VM) (see Refs. iKimura and Weiss 



(Il964l ): ICastellano et all t00$ )) 



In the VM, S = {0, 1} meaning that each agent is 
i = 1, . . . , N which takes a value among two pos- 



characterized by an attribute x 
sible alternatives. The set of all possible combinations of attributes of the agents is 
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S = {0, 1}^, that is, the set of all bit-strings of length N. At each time step in the 
iteration process, an agent i is chosen at random along with one of its neighboring 
agents j. If the states (xi,Xj) are not equal already, agent i adopts the state of j 
(by setting = Xj). At the microscopic level of all possible configurations of agents 
the VM corresponds therefore to an absorbing random walk on the TV-dimensional 
hypercube. It is well known that the model has the two absorbing states (1, . . . , 1) 
and (0, . . . , 0). For a system of three agents this is shown in FigJTJ 

b e 




Figure 1: The micro chain for the VM with 3 agents and its projection onto a random 
walk obtained by agglomeration of states with the same number of black agents b. 



Opinion models as the VM are a nice examples where our projection construction 
is particularly meaningful. There, we consider the projection IT?, that maps each 
x G S into Xf, G X where b is the number of agents in x with opinion 1. The 
projected configuration space is then made of the X^ where < b < N (see Fig{T|). 
Markov chain theory, in particular lumpability, allows us to determine conditions for 
which the macro chain on X = (Xq, . . . , Xb, . . . , Xn) is again a Markov chain. We 
find that this requires that the probability distribution oj must be invariant under the 
group Sn of all the permutations of N agents and therefore uniform. This underlines 
the theoretical importance of homogeneous or complete mixing in the analysis of the 
VM and related models. 

In this way our method enables the use of Markov chain instruments in the math- 
ematical analysis of ABMs. In Markov chains with absorbing states (and therefore in 
the ABM) the asymptotic status is quite trivial. As a result, it is the understanding 
of the transient that becomes the interesting issue. In order to analyze the transient 
dynamics for the macro dynami cs, all that is needed is to compute the fundamental 
matrix F of the Markov chain ( Kemeny and Snell ( 1976h ) . For the binary VM we 
are able to derive a closed form expression for the elements in F for arbitrary N 
which provides us with all the information needed to compute the mean quantities 
and variances of the transient dynamics of the model. In addition, we show in the 
VM with three opinion alternatives (S = {0,1,2}) how restrictions in communica- 
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tion (bounded confidence) lead to stable co-existence of different opinions because 
new absorbing states emerge in the macro chain (see Banisch et al. (2012)). 

3 Some Results 

The micro chains obtained via the random map representations help to understand 
the role of the collection of (deterministic) interaction rules used in the model from 
one side and of the probability distribution u governing the sequential choice of 
the rules used to update the system at each time step from the other side. The 
importance of this probability distribution is to encode social relations and exchange 
actions. In our setting it becomes explicit how the symmetries in u translate into 
symmetries of the micro chain. If we decide to remain at a Markovian level, then the 
partition, or equivalently the collective variables to be used to build the macro model 
must be compatible with the symmetry of the probability distribution u. In order 
to account for an increased level of heterogeneity the partition of the configuration 
space defining the macro-level has to be refined. A first result into this direction is 
that the symmetry group of agent permutations on u informs us about ensembles 
of agent configurations that can be interchanged without affecting the probabilistic 
structure of micro chain. Consequently, these ensembles can be lumped into the 
same macro state and the dynamical process projected onto these states is still a 
Markov chain. It is clear, however, that, in absence of any symmetry, there is no 
other choice than to stay at the micro-level because no Markovian description at a 
macro-level is possible in this case. 

In our opinion, a well posed mathematical basis for linking a micro-description of 
an ABM to a macro-description may help the understanding of many of the proper- 
ties observed in ABMs and therefore provide information about the transition from 
the interaction of individual actors to the complex macroscopic behaviors observed 
in social systems. We summarize our main results below: 

1. We formulate agent-based models as Markov chains at the micro level with 
explicit transition probabilities. 

2. This allows the use of lumpability arguments to link between the micro and 
the macro level. 

3. In case of a non-lumpable macro description this explains the emergence non- 
trivial dynamical effects (long memory). 

4. In the Voter Model, homogeneous mixing leads to a macroscopic Markov chain 
which underlines the theoretical importance of homogeneous mixing. 

5. This chain can be solved including mean convergence times and variances. 

6. The stable co-existence of different opinions with in the bounded confidence 
model follows from the emergence of new absorbing states in the macro chain. 
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7. Heterogeneous mixing requires refinement and we show how to exploit the 
symmetries in the mixing distribution (u) to obtain a proper refinement. 

Further Reading 

Banisch, S., Lima, R., and Araujo, T. (2012). Agent Based Models and Opinion Dynamics 
as Markov Chains, accepted by Social Networks, (jhttp: / / arxiv.org/abs/1108. 1716 ). 



Banisch, S. and Lima, R. (2012). Markov Projections of the Voter Model, unpublished, 
( http: / / www.universecity.de/MarkovProjections0312.pdf I . 
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